{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# About Sync Mode\n",
        "\n",
        "Some users may have such a request: they do not want to use the complex graph combination and connection of the BMF framework, but want to use the ability of a certain module alone. Sync Mode provides users with a function of directly invoking module capabilities in an atomic form. Users can directly call the functions of the module without building a graph, as shown in the following figure:\n",
        "\n",
        "This article mainly introduces the implementation of Sync Mode Python. Go and C++ also have corresponding implementation mechanisms:\n",
        "\n",
        "\n",
        "![未命名.png]()\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "F5VPsX1Lixae"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1. Install BMF in Python environment."
      ],
      "metadata": {
        "id": "MjaNzZ3fse4C"
      }
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "z9Jses5ssg9y"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-kyet1u_GYNc"
      },
      "outputs": [],
      "source": [
        "!pip install BabitMF"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2.The Engine layer of the BMF framework is uniformly implemented in C++ language. In Colab, when python calls the C++ library, the log of the C++ library layer will be hidden, so it is necessary to install and load the wurlitezer library to enable logs in the C++ layer."
      ],
      "metadata": {
        "id": "MtkenCr7syuG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install wurlitzer\n",
        "%load_ext wurlitzer"
      ],
      "metadata": {
        "id": "O1lA6mqGIC5m"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 3. Download a sample transcoded video (using Big Bunny as an example here)."
      ],
      "metadata": {
        "id": "3nyirq96tBuA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://github.com/BabitMF/bmf/releases/download/files/files.tar.gz\n",
        "!tar -zvxf files.tar.gz"
      ],
      "metadata": {
        "id": "GU5LOzPXE1GW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 4. Write and implement a sync mode calling code.\n",
        "### This piece of code first calls the bmf_sync.sync_module interface to create four bmf modules (c_ffmpeg_decoder, c_ffmpeg_filter (Scale), c_ffmpeg_filter (volume), c_ffmpeg_encoder). Afterwards, it continuously reads the video stream from the input video by loop, decodes frame by frame, and first sends it to a Scale Filter Module to scale the video to a resolution of 320x250. Then, the processed video frame is obtained and sent to a volume Filter Module to adjust the volume once. Finally, the video is sent to the Encoder Module for video encoding and saved as a file. Let's complete the demo implementation of sync mode through two sub-steps"
      ],
      "metadata": {
        "id": "wEYaqf_ct5Oa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.1 Creating Sync Modules\n",
        "In this code, we first called the bmf.sync_module interface to create three modules. The definition of this interface follows as below.\n",
        "\n",
        "```\n",
        "def bmf.builder.bmf_sync.sync_module (\n",
        "   name,\n",
        "   option,\n",
        "   input_streams,\n",
        "   output_streams\n",
        ")\n",
        "\n",
        "Create SyncModule by name, option, input_stream_id_list and output_stream_id_list.\n",
        "\n",
        "Parameters\n",
        "\n",
        "name the name for the module\n",
        "option the option for the module\n",
        "input_streams the input stream id list for the module\n",
        "output_streams the output stream id list for the module\n",
        "\n",
        "Returns\n",
        "bmf_sync.SyncModule\n",
        "\n",
        "```\n",
        "\n",
        "Using the interface above, we created one Decoder module, two Filter modules, and one Encoder module. For the Decoder module, we set up two output streams (encoded as number 0 and number 1), where number 0 corresponds to the video stream and number 1 corresponds to the audio stream. For the two Filter modules, Scale and Volume, we set up one input stream and one output stream each, with a unified number of 0. Finally, the Encoder module takes in two input streams and has no output stream, as the Encoder module will encode the processed video stream and save it to disk.\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "QvST_QyjdQgV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import bmf\n",
        "from bmf import bmf_sync, Packet\n",
        "input_video_path = \"./files/big_bunny_10s_30fps.mp4\"\n",
        "output_path = \"./video.mp4\"\n",
        "\n",
        "# create sync modules\n",
        "decoder = bmf_sync.sync_module(\"c_ffmpeg_decoder\", {\"input_path\": input_video_path}, [], [0, 1])\n",
        "scale = bmf_sync.sync_module(\"c_ffmpeg_filter\", {\n",
        "    \"name\": \"scale\",\n",
        "    \"para\": \"320, 250\"\n",
        "}, [0], [0])\n",
        "volume = bmf_sync.sync_module(\"c_ffmpeg_filter\", {\n",
        "    \"name\": \"volume\",\n",
        "    \"para\": \"volume=3\"\n",
        "}, [0], [0])\n",
        "encoder = bmf_sync.sync_module(\"c_ffmpeg_encoder\", {\n",
        "    \"output_path\": output_path\n",
        "}, [0, 1], [])\n"
      ],
      "metadata": {
        "id": "vM3JZjmXHO-k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4.2 Writing and Implementing Pipeline Code\n",
        "\n",
        "In the process described above, we have built the required 4 syncModules, let's write a pipeline to implement a video processing process as shown in the figure below:\n",
        "\n",
        "![sync_module_pipeline.drawio.png]()\n",
        "\n",
        "When we build the pipeline, we mainly call the bmf_sync.process interface, which should be defined as follows:\n",
        "\n",
        "\n",
        "```\n",
        "def bmf.builder.bmf_sync.process (\n",
        "  module,\n",
        "  pkts_dict\n",
        ")\n",
        "\n",
        "Directly do module processing.\n",
        "\n",
        "Parameters\n",
        "\n",
        "module corresponding syncModule object\n",
        "pkts_dict a dict which contains all input data packet\n",
        "\n",
        "Returns\n",
        "result_dict, task.timestamp\n",
        "```\n",
        "Using this interface, we combined the modules created above and used a dictionary to map the input pkt to the InputStream and OutputStream of the syncModule one by one, then implemented the entire video processing pipeline.\n",
        "\n"
      ],
      "metadata": {
        "id": "vjum6iRAeS6a"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# process video/audio by sync mode\n",
        "while True:\n",
        "    frames, _ = bmf_sync.process(decoder, None)\n",
        "    has_next = False\n",
        "    for key in frames:\n",
        "        if len(frames[key]) > 0:\n",
        "            has_next = True\n",
        "            break\n",
        "    if not has_next:\n",
        "        bmf_sync.send_eof(encoder)\n",
        "        break\n",
        "    if 0 in frames.keys() and len(frames[0]) > 0:\n",
        "        frames, _ = bmf_sync.process(scale, {0: frames[0]})\n",
        "        bmf_sync.process(encoder, {0: frames[0]})\n",
        "    if 1 in frames.keys() and len(frames[1]) > 0:\n",
        "        frames, _ = bmf_sync.process(volume, {0: frames[1]})\n",
        "        bmf_sync.process(encoder, {1: frames[0]})"
      ],
      "metadata": {
        "id": "DSXwk_A8eSV-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 5. Display the video streams before and after processing."
      ],
      "metadata": {
        "id": "nzn35ZPZvhyR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import HTML\n",
        "from base64 import b64encode\n",
        "\n",
        "def show_video(video_path, video_width = 800):\n",
        "  video_file = open(video_path, \"r+b\").read()\n",
        "  video_url = f\"data:video/mp4;base64,{b64encode(video_file).decode()}\"\n",
        "  return f\"\"\"\n",
        "  <video width={video_width} controls>\n",
        "    <source src=\"{video_url}\">\n",
        "  </video>\n",
        "  \"\"\"\n",
        "\n",
        "video_url1 = show_video('./files/big_bunny_10s_30fps.mp4')\n",
        "video_url2 = show_video('video.mp4')\n",
        "\n",
        "html = video_url1 + video_url2\n",
        "HTML(html)"
      ],
      "metadata": {
        "id": "zACHQOdMvbEZ"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}